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from  the  director  . . . 


As  you  read  this  edition  of  the  Resource ,  I  hope  that 
you  are  as  impressed  as  I  am  at  the  never-ending 
stream  of  energy  depicted  herein.  I  see  the  full  gamut 
of  dedicated  professional  people  that  are  represented 
by  this  publication — some  represented  explicitly  and 
some  implicitly  All  of  the  individuals  are  necessary  for 
the  success  of  this  program. 

Be  impressed  by  the  technical  accomplishments,  the 
vision  of  tomorrow,  the  dedication  to  improve  the  usef  s 
experience,  and  the  planning  for  the  future. 

The  work  being  done  inAirblast  Technology  is  truly 
intriguing.  The  use  of  the  Cray  XT3  to  leverage  insight 
into  High-Power  Microwave  technologies  is  awesome. 
Timely  and  inspiring  is  how  I  would  describe  the 
constant  improvement  to  techniques  and  algorithms 
displayed  in  the  sorting  of  ocean  wave  data  and  the 
application  of  user  toolkits  to  help  scientists  focus  on 
science  and  not  on  how  to  use  the  tools. 

While  absorbing  the  technological  improvements,  don’t 
miss  the  well-deserved  recognition  for  John  West’s 
visionary  contributions  to  the  MSRC  and  the  high 
performance  computing  community.  Take  heed.  Pay 
attention.  I  am  sure  there  will  be  more. 

The  User  Interface  Toolkit,  ezVIZ,  and  the  update  to 
the  MSRC  Web  site  are  all  examples  of  how  we  are 
trying  to  keep  the  support  for  our  users  fresh  and  up  to 
date. 

Continuing  through  you  will  notice  that  even  though  we 
are  not  upgrading  one  of  our  major  systems,  we 
continue  to  upgrade  our  MSRC  infrastructure.  Com¬ 
munications  is  just  one  area  gettingattention.  Planning 
is  being  done  to  make  sure  that  we  are  prepared  for  the 
continuing  technological  upgrades.  Planning  is  also 
being  done  with  another  resource,  people.  Inside  this 
edition  you  will  see  examples  of  how  investing  in  our 
youth  plays  out  to  help  secure  a  strong  future  for  the 
MSRC. 


Greg  Rottman 
Acting  Director,  ERDC  MSRC 


Speaking  of  people,  how  much  of  this  would  be  accom¬ 
plished  without  them.  Good  people.  Professional 
people.  As  you  read  these  articles,  think  of  the  people 
that  work  so  hard  to  make  all  of  this  possible.  Who  is 
the  one  that  makes  sure  there  is  fuel  in  the  backup 
generator  when  it  is  needed?  Who  is  the  one  that  plans 
for  the  power  we  need  for  the  computer  system,  the  air 
conditioners,  the  network  devices?  Who  is  the  one  that 
makes  sure  the  maintenance  contracts  are  renewed  on 
time  so  that  the  system  can  be  repaired?  Who  is  the 
one  that  makes  the  funding  available? 

Just  like  updating  algorithms  and  computer  systems, 
there  is  a  continual  need  for  checking  the  infrastructure 
strength  and  potential  requirements.  And  just  like  the 
technical  and  scientific  work,  the  only  way  this  will 
ever  get  accomplished  is  with  dedicated  individuals 
working  as  a  team. 

How  does  the  saying  go?  It  takes  a  village  to  run  a 
supercomputer? 

As  always  please  let  us  know  how  we  are  doing.  My 
e-mail  address  is  Greg.K.Rottman@erdc.usace. 
army.mil. 


Greg  Rottman 

Acting  Director  ERDC  MSRC 


About  the  Cover:  Cover  design  by  the  ERDC  MSRC  Scientific  Visualization  Center  For  related  stories,  see 
pages  4  and  22. 


Person  to  Watch  —  John  E.  West 

By  Rose  J.  Dykes 

While  serving  as  Director  of  the  ERDC  High  Performance 
Computing  (HPC)  Major  Shared  Resource  Center  (MSRC), 

John  West  was  chosen  as  one  of  the  HPCwire  “People  to 
Watch  for  2006.”  According  to  Tom  Tabor,  HPCwire  pub¬ 
lisher,  this  annual  list  “recognizes  the  most  infhential,  interest¬ 
ing  and  promising  personalities  having  an  impact  oithe  world 
of  High  Performance  Computing.”  He  also  says,  “the  names 
on  this  list  have  been  nominated  by  a  selection  committee  that 
includes  past  “People  to  Watch”  as  well  as  a  long  list  of 
advisors  for  the  publication.”  Find  out  more  about  the  selec¬ 
tions  in  the  article  entitled  “ HPCwire  s  People  to  Watch  2006,” 
which  can  be  found  athttp://www. taborcommunications.com/hpcwire/features/people06/index.html.  The  article 
states  that  John  “is  worth  watching  because  of  his  visionary  goals  for  making  HPC  accessible  to  new  and  diversi¬ 
fied  user  groups.”  HPCwire  is  considered  a  leading  source  for  global  information  on  HPC. 

People  at  Mississippi  State  University  (MSU),  John’s 
alma  mater,  have  kept  their  eyes  on  John  and 
recently  asked  him  to  speak  at  their  HPC  Colloquium 
at  MSU.  While  participating  in  the  Colloquium,  he 
also  attended  the  HPC  Collaboratory  (HPC2) 

Building  Addition  Grand  Opening,  at  which  Missis¬ 
sippi  Governor  Haley  Barbour  spoke.  HPC2  is  a 
premier  institute  for  advancement  of  computational 
engineering  and  comprises  resources  in  computing 
facilities  and  computational  expertise  that  are 
recognized  internationally. 

ERDC  folks  have  also  been  watching  John  and 
recently  named  him  the  Acting  Deputy  Director  of 
the  ERDC  Information  Technology  Laboratory 
(ITL),  one  of  seven  unique  laboratories  comprising 
ERDC.  ERDC  R&D  supports  the  Department  of 
Defense  and  other  agencies  in  military  and  civilian 
projects  and  was  named  the  Army’s  Large  R&D 
Organization  of  the  Year  in  2005. 


John  assumed  his  new  role  on 
September  2  after  Dr .  Jeffery  P. 
Holland,  former  ERDC  ITL 
Director,  was  named  Acting 
Deputy  Director  of  ERDC,  and 
Dr.  Deborah  Dent,  former  ERDC 
ITL  Deputy  Director,  was  named 
Acting  ERDC  ITL  Director. 


ERDC  MSRC  honors  John 
at  a  surprise  celebration  with  the 
theme  “We  are  Watching  You ” 


John  speaks  at  Mississippi  State  University’s  High 
Performance  Computing  Colloquium  on  August  30 
(photograph  courtesy  of  Dr  Fred  Tracy) 


John  E.  West — dubbed  by  HPCwire  in  May  as  a 
person  to  watch  for  2006 
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ERDC’s  New  Commander  Visits  ITL  and  ERDC  MSRC 

By  Rose  J.  Dykes 


Colonel  Richard  B.  Jenkins’  visit  to  the  ERDC 
Information  Technology  Laboratory  included  a  tour 
through  the  ERDC  MSRC  High  Performance 
Computing  Center  and  ScientificVisualization 
Center  on  September  8.  He  assumed  command  of 
ERDC  on  July  6,  2006.  In  thanking  ITL  team 
members  afterwards,  he  wrote,  “It  was  great 
spending  time  with  the  Lab  and  quite  eye  opening  to 
learn  more  about  the  superb  work  you  all  are  doing. 

I  look  forward  to  spending  more  time  with  ITL.” 

Prior  to  assuming  command  of  ERDC,  COL  Jenkins 
was  the  District  Engineer  and  Commander  of  the 
Gulf  Region  Division,  North  Districtin  Mosul,  Iraq, 
and  responsible  for  management  and  quality  control 
of  billions  of  dollars  in  reconstruction  and  new 
construction  throughout  Iraq.  His  responsibilities 
ranged  from  critical  infrastructure  for  essential 
utility  services  such  as  reliable  electric  power  and 
clean  water,  to  smaller  yet  vital  projects  to  provide 
quality  healthcare,  transportation,  and  education 
facilities. 


Dr.  Deborah  Dent  (lelt),  Acting  ERDC  ITL  Director, 
welcomes  COL  Richard  B.  Jenkins  to  ITL 
(photograph  courtesy  of  Marsha  Gay) 


(Seated  left  to  right)  COL  Jenkins,  Greg  Rottman,  Acting  ERDC 
MSRC  Director,  and  Richard  Walters,  ERDC  MSRC  Scientific 
Visualization  Center,  and  John  l/l bst  (standing), 

Acting  ERDC  ITL  Deputy  Director 


Before  his  assignment  to  Iraq,  COL  Jenkins 
served  as  Deputy  Commandei;  Mississippi 
Valley  Division,  and  Secretary  Mississippi 
River  Commission.  During  thisassignment, 
he  deployed  in  2003  as  Deputy  Commander 
and  Chief  of  S  taff  for  Task  Force  Restore 
Iraqi  Oil. 

COL  Jenkins  graduated  from  the  U.S. 
Military  Academy  at  West  Point  and  has 
served  in  a  variety  of  assignments  in  the 
United  States  and  Europe.  His  military 
education  includes  the  Engineer  Officer 
Basic  and  Advanced  Courses;  Combined 
Arms  Services  Staff  School;  Command  and 
General  Staff  College;  and  the  Naval  War 
College.  Jenkins  earned  master’s  degrees  in 
Civil  Engineering  from  Cornell  University 
and  in  National  Security  and  Strategic 
Studies  from  the  U.S.  Naval  War  College. 
He  is  a  registered  professional  engineer  in 
the  Commonwealth  ofVirginia. 
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Computational  and  Experimental  Studies  of  Airblast 
in  Urban  Terrain 

By  Byron  J.  Armstrong,  Dr.  James  T.  Baylot,  and  Denis  D.  Rickman 
ERDC  Geotechnical  and  Structures  Laboratory 


Protecting  United  States  military  personnel  from 
terrorist  attacks  is  critical  to  the  U.S.  Department  of 
Defense  (DoD)  mission  success  as  indicated  by  the 
selection  of  Combating  Terrorism  as  a  Joint  Warfighting 
Capability  Objective.  Real-life  scenarios  rarely  consist 
of  simply  an  explosion,  the  propagation  of  the  blast  load 
over  a  flat  terrain,  and  its  impact  on  a  structure  of 
interest.  The  blast  environment  is  affected  by  the 
presence  of  other  structures,  as  well  as  the  natural 
topography.  A  complete  understanding  of  this  complex 
blast  environment,  including  the  effects  of  adjacent 
structures  on  the  blast  loads  on  a  structure,  is  needed  to 
protect  building  occupants  in  an  urban  environment. 
Underpredicting  the  loads  will  lead  to  an  underassess¬ 
ment  of  the  risk  and  to  possible  underdesign  of  retrofits 
to  provide  protection  from  that  risk.  Overpredicting  the 
loads  could  lead  to  investing  funds  in  the  wrong  location, 
or  worse,  deciding  that  it  is  not  worth  the  investment. 

The  ERDC  Geotechnical  and  Structures  Laboratory 
(GSL)  is  the  lead  DoD  laboratory  for  research  in  the 
civil  engineering  area  of  Survivability  and  Protective 
Structures  (S&PS).  The  S&PS  research  program  is 
focused  on  the  warfighter ’s  needs  for  force  protection 
and  counterterrorist  threats.  In  the  execution  of  its 
missions,  the  Army  will  be  challenged  by  increasingly 
advanced  conventional  and  terrorist  weapons  threats 
that  will  be  readily  available  to  many  of  the  nations  in 
the  world.  The  threat  from  terrorist  or  saboteur  weap¬ 
ons  must  be  countered  during  peacetime  as  well  as 
during  military  operations.  The  ability  to  ddermine 
weapons  effects  and  predict  the  survivability  of  struc¬ 
tures  is  essential  for  survival  of  the  Army. 

The  GSL  is  using  a  combination  of  small-scale  experi¬ 
ments  and  numerical  simulations  to  study  the  blast  in 
urban  terrain.  The  experiments  are  used  to  collect  data 
to  study  the  phenomenology,  as  well  as  to  validate  the 
numerical  simulations.  The  numerical  simulations  are 
used  to  assist  in  designing  the  experiments,  to  augment 
the  data  from  the  experiments,  and  to  visualize  the  blast 
propagation  in  the  experiments.  Simulations  are  also 
used  to  collect  “data”  for  experiments  that  either 
cannot  or  will  not  be  conducted. 

The  GSL  has  conducted  several  series  of  small-scale 
experiments  (Ligure  1)  to  study  the  effect  of  adjacent 
buildings  on  blast  loadings.  In  the  first  year  of  its  blast- 
in-urban-terrain  studies,  the  GSL  conducted  two- 


building  experiments  and  simulations.  These  are  aimed 
at  studying  the  shielding  effects  of  one  building  on  the 
other.  The  explosive  charge  was  detonated  in  front  of 
one  of  the  buildings,  and  pressure  measurements  were 
made  on  the  front,  back,  side,  and  top  of  the  building 
closest  to  the  explosives  and  on  the  shielded  building. 
Companion  single-building  experiments  and  simulations 
were  performed  for  a  direct  comparison  with  the  two- 
building  experiments.  The  next  yeai;  three-building 
experiments  and  simulations  were  performed,  and  five- 
building  simulations  were  performed.  Finally  in  the  third 
year,  nine-building  portions  of  a  typical  city  layout  were 
studied. 

Numerical  simulations  were  performed  to  model  the 
configurations  that  were  tested  and  other  configurations 
that  were  not  tested.  The  research  was  conducted 
using  an  allocation  of  the  DoD  High  Performance 
Computing  Modernization  Program  Office  (HPCMPO) 
Challenge  processing  hours  at  the  ERDC  MSRC  under 
challenge  project  C83  “Evaluation  and  Retrofit  for 
Blast  in  Urban  Terrain.”  The  Second-order  Hydrody¬ 
namic  Automatic  Mesh  Refinement  Code  (SHAMRC) 
was  used  for  the  simulations.  SHAMRC  is  an  Eulerian 
finite  difference  code  refined  for  the  explicit  purpose  of 
calculating  airblast  propagation.  First-year  simulations 
were  performed  on  the  IBM  SMP  at  the  Aeronautical 
Systems  Center  (ASC)  MSRC.  Second-  and  third-year 
simulations  were  performed  on  the  Compaq  SC40  and 
SC45  systems  at  the  ERDC  MSRC. 

The  two-building  model  consisted  of  about  38  million 
cells.  The  25  msec  simulation  required  160  hours  on 
23  processors  (3,680  processor  hours)  to  complete  the 
simulation.  The  three-bulding  model  consisted  of 
approximately  38  million  cells.  The  5  msec,  three- 
building  simulation  required  540  GBytes  disk  space  and 
4,100  processor  hours  on  32  processors.  In  the  five- 
building  simulation,  data  for  animations  were  not  saved. 
This  significantly  reduced  the  disk  space  required.  The 
5  msec  simulation  of  40  million  cells  was  acomplished 
in  527  processor  hours  on  64  processors.  A  total  of 
12.6  square  meters  was  modeled  in  this  simulation.  The 
nine -building  simulations  were  performed  in  three  steps. 
In  the  first  step,  the  explosive  bum  was  computed  using 
a  two-dimensional  axisymmetric  model  with  very  fine 
variable  cell  spacing.  The  second  model  carries  the 
shock  past  the  nearest  buildings.  This  model  has  a  cell 
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(a)  Single  building  (b)  Two  buildings 


(c)  Three  buildings 


(d)  Five  buildings 


Figure  1.  Initial  multiple  building  studies.  Fringe  plots  on  front  of  large  building  of  ratio  of  peak  pressure  to  peak 

pressure  at  same  location  in  single-building  simulation 


spacing  of  0.4  cm  and  had  about  75  million  cells. 
Finally,  the  cell  spacing  is  changed  to  0.8  cm,  and  89 
million  cells  model  the  entire  geometry  Each  20  msec 
simulation  required  1,144  GBytes  disk  space  and  11,000 
processor  hours  on  64  processors. 

The  numerical  simulations  provide  a  more  complete 
understanding  of  both  shielding  and  focusing  effects. 
Figure  1  compares  peak  pressure  ratios  for  the  two-, 
three-,  and  five-building  simulations.  Peak  pressures 
were  determined  for  a  number  of  locations  on  the  side 
of  the  large  building  closest  to  the  explosive.  At  each 
location  the  peak  pressure  was  normalized  by  dividing 
by  the  peak  pressuie  for  the  single-building  simulation 
(Figure  la).  Experimental  data  were  collected  at  five 
heights  on  the  center  line  of  the  large  building. 

The  figure  shows  the  expected  results  for  the  two- 
building  simulation.  Peak  pressures  at  all  locations  are 
less  than  or  equal  to  the  peak  pressures  for  the  single¬ 


building  simulation.  Near  the  top  center  the  pressures 
are  nearly  equal  to  the  single-building  simulations. 

These  locations  are  above  the  height  of  the  small 
building  that  is  shielding  it.  Near  the  bottom  of  the  large 
building,  the  peak  pressure  is  between  25  and  50 
percent  of  the  single-building  value. 

The  three-building  results  illustrate  both  focusing  and 
shielding  effects.  Near  the  center,  the  pressures  are 
much  higher  than  in  the  single-building  simulation.  Near 
the  outer  edges,  the  blast  is  somewhat  shielded  by  the 
two  smaller  buildings,  and  the  pressures  are  lower  than 
in  the  single-building  simulation.  In  the  five-building 
simulation,  the  explosive  charge  is  located  between  two 
of  the  structures,  and  the  focusing  effects  on  the  front 
of  the  large  building  are  severe.  These  focusing  effects 
overcome  the  shielding  effects,  and  the  peak  pressures 
over  most  of  the  front  of  the  large  building  significantly 
exceed  the  single-building  values. 
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The  simulations  show  that  there  are  differences  in 
shielding  and  confining  effects  when  centerline  values 
are  compared  with  those  away  from  the  center .  Since 
instruments  were  not  provided  at  these  locations,  the 
experiments  could  not  have  illustrated  these  effects. 

The  five-building  simulation  shows  significantly  en¬ 
hanced  confinement  effects.  The  five-building  results 
are  based  on  the  validated  numerical  model.  These  two 
results  demonstrate  the  value  of  the  numerical  simula¬ 
tion  in  discovering  effects  that  could  not  have  been 
found  using  the  experiments  only 

One  of  the  nine -building  configurations  simulated  and 
tested  in  the  last  year  of  the  program  is  shown  in  Figure 
2.  The  explosive  charge  is  detonated  between  buildings 
2  and  5  (red  dot  near  center  of  experimental  photo). 
Figure  2  also  compares  the  simulated  pressure  history 
with  the  average  of  the  pressure  histories  from  the 
three  experiments  tested  for  this  configuration.  The 
gauge  was  located  on  the  back  of  building  5  halfway 
across  the  back  and  near  the  ground  surface.  Pres¬ 
sures  and  times  are  scaled  by  the  same  values  for  the 
simulation  and  the  experiment.  The  figure  shows  good 
comparison  between  the  experiment  and  the  simulation. 
The  simulation  has  slightly  delayed  arrival  times,  and 
the  peak  pressure  for  the  second  major  pressure  pulse 
is  slightly  higher  in  the  simulation. 

Pressure  fringe  plots  at  critical  times  affecting  the  loads 
on  the  back  of  building  5  are  shown  in  Figure  3.  Some 
of  the  complex  interactions  of  the  blast  can  be  followed 
by  examining  these  fringe  plots.  The  initial  pressure 
pulse  shown  in  Figure  2  is  the  arrival  of  the  direct 
shock  propagating  around  building  5  at  a  scaled  time  of 
0.48  and  is  not  affected  by  the  other  buildings.  The 
highest  shock  pressure  peak  in  Figure  2  is  significantly 
affected  by  the  reflections  off  buildings  2,  4,  6,  and  8. 
The  effects  of  all  of  these  reflections  reach  the  back  of 
building  5  at  a  scaled  time  of  1 .0 

Significant  improvements  to  SHAMRC  were  made 
during  this  program.  Joseph  Crepeau  ofApplied 
Research  Associates  (the  current  developer  of 
SHAMRC)  was  very  helpful  in  getting  versions  of 
SHAMRC  executing  on  the  MSRC  systems  used  for 
this  study.  Paul  Adams  and  David  Longmire  of  the 
ERDC  MSRC  Scientific  Visualization  Center  were 
very  helpful  in  visualizing  the  results  of  the  simulations. 
The  research  reported  herein  was  conducted  as  part  of 
the  U.S.  Army  Corps  of  Engineers  Survivability  and 
Protective  Structures  Technical  Area  by  the  Weapons 
Effects  and  Structural  Response  Work  Package,  Work 
Units  AT40-WE-001  “Development  of  Improved 
Airblast  Predictive  Methods”  and  AT40-WE-004  “HPC 


Applications  for  Blast  Hardening.”  Permission  from 
the  Director,  Geotechnical  and  Structures  Laboratory, 
to  publish  this  article  is  gratefully  acknowledged.  The 
research  was  conducted  using  an  allocation  of 
HPCMPO  Challenge  processing  hours  at  the  ERDC 
and  ASC  MSRCs. 


Figure  2.  Scaled  pressure  history  comparison  with 
average  of  three  experiments  near  ground  on  back  of 
building  5  for  detonation  between  buildings  2  and  5 


6 


ERDC  MSRC  Resource,  Fall  2006 


m 


Scaled  time  0.74  Scaled  time  0.97  Scaled  time  1.0 


Figure  3.  Pressure  fringe  plots  on  ground  plane  at  criticatimes  for  detonation 

between  buildings  2  and  5 
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ERDC  Cray  XT3  Supercomputer  Aids  Air  Force 
Researchers  in  High-Power  Microwave  Technologies 

By  Jeff  MacGillivray,  U.S.  Air  Force  Research  Laboratory 


Large  memory  capacity,  fast  processor  speed,  and  the 
large  number  of  processors  of  the  Cray  XT3 
supercomputer,  located  at  the  U.S.  Army  Engineer 
Research  and  Development  Center  (ERDC), 

Vicksburg,  MS,  have  recently  been  just  what  the  U.S. 
Air  Force  Research  Laboratory  scientists  needed. 
Preliminary  results  have  yielded  good  data  for  the  study 
of  complex  temporal  electromagnetic  problems. 

Air  Force  scientists  involved  in  the  project  wanted  to 
simulate  new  antenna  designs  and  field  interaction 
problems.  Once  validation  of  the  antenna  simulations  is 
accomplished,  modeling  can  then  be  used  to  assess 
various  antenna  configurations  without  costly  construe 
tion  and  experiments.  Field  interaction  modeling  pro 
vides  animations  revealing  how  fields  penetrate  into 
objects. 

The  Air  Force  Research  Laboratory  has  developed  an 
electromagnetic  finite  difference  time  domain  code 
TMax  (Temporal  Maxwell  solver)  for  Department  of 
Defense  (DoD)  application  problems.  TMax  is  a  robust 
solver  that  harnesses  massive  parallel  and  vector 
processing  to  a  high  degree  of  efficiency  requiring  little 
maintenance,  thereby  providing  a  powerful  electromag 
netic  modeling  tool.  By  combining  CAD/mesh  genera 
tion  tools  for  ultra  high-resolution  meshes  with  ever 
increasing  DoD  High  Performance  Computing  Mod¬ 
ernization  Program  (HPCMP)  processor  speeds  and 
available  memory,  simulations  of  complex  shapes  and 
sizable  volumes  are  easily  realized. 

The  first  attempt  at 
running  TMax  on  the 
ERDC  Cray  XT3  was 
completely  successful. 

Fifteen  billion  cells  on 


needed,  30  billion  TMax  cells  will  fit  onto  1,000  proce-s 
sors  or  60  billion  on  2,000  processors.  Over  708  million 
cells  on  only  12  processors  ran  on  the  XT3  in  10  hours, 
also  for  one  incident  field  pass  across  the  mesh. 

The  ERDC  MSRC  Scientific  Visualization  Center  aided 
by  producing  animations  (see  image  below).  The 
animations  will  provide  information  for  experiments  that 
will  indicate  how  much  time  is  needed  to  reach  steady 
state  and  field  intensity  on  an  automobile  with  occu 
pants.  Investigations  will  also  be  made  on  more  com 
plex  questions. 

Field  interaction  simulations  make  good  physical  sense 
and  allow  the  experimenter  to  observe  temporal 
penetration  into  various  objects  via  animations.  These 
results  demonstrate  that  a  progressively  larger  number 
of  complex  temporal  electromagnetic  problems  can  be 
solved  with  HPMCP  computational  platforms  by 
developing  software  that  efficiently  harnesses  high 
performance  computing  resources. 

High-power  microwaves  have  significant  potential  in 
command  and  control  warfare.  The  low-collateral 
damage  aspect  of  high-power  microwave  weapons 
makes  them  useful  in  a  wide  variety  of  missions  where 
avoiding  casualties  is  a  major  concern.  A  short  burst  of 
high-power  microwave  energy  can  be  lethal  to  elec¬ 
tronics  while  having  no  affect  on  humans  operating  the 
equipment. 


Image  shows  results  of  TMax  simulation  of  high-power  microwaves 
impacting  automobile 


1,000  processors  ran  in 
4  hours  for  one  incident 
field  pass  across  the 
volume  domain.  Nearly 
every  1,000-processor 
run  was  successful.  If 
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Sorting  Out  Waves  with  a  Fast  Sort  Algorithm 

By  Dr.  Fred  T.  Tracy,  ERDC  MSRC,  ITL,  Vicksburg,  Mississippi;  Barbara  A.  Tracy,  ERDC  Coastal  and 
Hydraulics  Laboratory  (CHL),  Vicksburg,  Mississippi;  and  Dr.  Jeffrey  L.  Hanson,  CHL  Field  Research 
Facility,  Duck,  North  Carolina 


Introduction  to  Ocean  Waves 

Knowing  what  the  ocean  is  going  to  do  has  been  an 
essential  part  of  military  planning  and  execution  since 
the  beginning  of  time.  Wave  forecasting  using  numeri¬ 
cal  methods  began  during  World  War  II  and  provided 
information  for  theAllied  invasion  of  Normandy  in  1944 
(U.S.  Army  Corps  of  Engineers  2002).  Knowledge 
about  the  behavior  of  oceans  is  also  important  for 
coastal  design,  harbors,  coastal  structures,  and  erosion 
control.  That  quickly  brings  the  discussion  to  a  need  to 
understand  ocean  waves.  Waves  develop  from  wind 
blowing  over  the  water.  Waves  can  be  very  compli¬ 
cated  as  demonstrated  by  Figure  1  (photograph  cour¬ 
tesy  of  Dr.  Fred  Tracy)  showing  part  of  the  “Twelve 
Apostles”  off  the  coast  of  Australia. 


Wave  Information  Study 

The  Wave  Information  Study  (WIS)  project  conducted 
under  the  Coastal  Field  Data  Collection  Program  in  the 
ERDC  Coastal  and  Hydraulics  Faboratory  maintains  a 
Web-based  database  for  ocean  wave  information  for 
United  States  coastlines  ((B.  Tracy  2006a).  Recently, 
the  parallel  version  of  WAVEWATCH  III  (Tolman 
2002a,  2002b)  was  used  on  the  ERDC  MSRC  Origin 
3900  to  model  the  Pacific  Ocean  to  create  a  10-year 
hindcast  for  the  WIS  Web  site.  The  purpose  of  the 
hindcast  is  to  closely  match  the  known  measured  data 
to  fill  in  unknown  data  everywhere  else  in  the  computa¬ 
tional  domain. 
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Working  in  Frequency  Space 

Figure  2  shows  a  time  series  plot  of  readings  from  the 
Gulf  of  Mexico  National  Data  Buoy  Center  (NDBC) 
buoy  42040  during  Hurricane  Katrina  (National  Oce¬ 
anic  and  Atmospheric  Administration  2006)  where  the 
significant  wave  height  and  dominant  period  (reciprocal 
of  frequency)  for  a  given  time  is  plotted.  Important  to 
note  is  that  the  maximum  height  at  this  particular  buoy 
was  55  ft  (16.76  m),  an  NDBC  record.  The  significant 
wave  height  and  dominant  period  are  bulk  parameters 
that  describe  the  frequency-direction  energy  spectrum 
at  this  time  slice  and  location  in  the  ocean. 

Often,  a  more  sophisticated  representation  of  wave 
data  is  needed  that  involves  working  in  frequency 
space.  Thus,  tools  like  the  Fast  Fourier  Transform  are 
commonly  used.  Wave  spectral  partitioning  methods 
have  been  developed  to  analyze  the  results  (Hanson 
and  Phillips  2001).  Figure  3  (Hanson  et  al.  2006)  shows 
one  such  representation  of  data  from  a  buoy  near 
Christmas  Island  in  the  Pacific  Ocean.  Here,  energy 


density  as  a  function  of  frequency  and  direction  is 
displayed.  The  swells  are  enhanced  energy  areas  in  the 
two-dimensional  frequency-direction  wave  spectrum 
coming  from  distant  sources  other  than  the  local  wind 
sea  region. 

Another  way  of  visualizing  partitioned  wave  data  is 
shown  in  Figure  4  (B.  Tracy  et  al.  2006b).  It  consists  of 
contours  of  the  energy  density  overlaid  onto  a  partition¬ 
ing  of  the  grid  for  a  hindcast  output  station  near  Oahu, 
Hawaii,  in  the  Pacific  Ocean.  The  partitioning  is  based 
on  a  watershed  algorithm  (Mncent  and  Soille  1991)  first 
prototyped  by  Hanson  and  Jensen  (2004)  for  the 
analysis  of  ocean  wave  data.  The  continental  divide 
where  everything  to  the  east  goes  into  the  Atlantic 
Ocean  and  everything  to  the  west  goes  into  the  Pacific 
Ocean  is  a  typical  example  of  a  watershed  line.  How¬ 
ever,  this  concept  of  partitioning  can  be  used  in  a  more 
abstract  sense  as  in  analyzing  frequency-direction  data 
from  a  spatial  grid. 
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Figure  2.  Time  series  plot  of  readings  from  NDBC  buoy  42040  during  Humane  Katrina 
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Partitioned  spectrum  for  2003,  November  21,  hour  21 


Frequency  numbers 

Figure  4.  Contour  and  wave  partition  plot 
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Watershed  Algorithm 

Determining  the  lines  separating  different  watersheds 
can  be  done  by  brute  force;  but  since  any  grid  point 
in  a  large  ocean  grid  (381  x  257)  for  a  parallel 
WAVE  WATCH  III  computation  may  need  the  compu¬ 
tation  and  visualization  shown  in  Figure  4,  a  quicker 
algorithm  is  needed.  The  improved  watershed  algorithm 
consists  of  four  steps:  (1)  compute  the  reciprocal  of 
each  term  of  the  energy  density  matrix  so  the  valleys 
become  ridges  or  watershed  lines;  (2)  round  all  these 
new  values  to  integers  between  1  and  ihmax  (100  used 
in  this  work)  representing  a  height;  (3)  sort  the  integer 
heights  in  ascending  order;  and  (4)  start  at  the  bottom 


and  advance  the  current  height  one  at  a  time,  assigning 
partition  values  as  climbing  continues.  The  end  result  is 
a  labeled  grid  with  each  grid  point  having  a  watershed 
or  partition  value  (three  in  Figure  4).  The  grid  points  at 
the  very  “top  of  the  mountain”  are  often  given  a  value 
of  one  of  their  neighbors  as  was  done  in  Figure  4. 

Fast  Sort  Algorithm 

After  the  integer  height  image  values  have  been  placed 
into  a  single  array,  sorting  the  data  in  ascending  order 
can  also  be  very  time-consuming  if  done  by  brute  force 
(0(N2)).  Figure  5  shows  an  algorithm  (F.  Tracy  2006) 
that  reduces  this  sort  to  0( N).  The  algorithm  implements 


c 

C 

c 

c 

C 

C 

c 

c 

c 

o 

c 

C 

c 

c 

c 

C 

c 

C 

C 
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Input  Variables 


ihmax  Largest  integer  height  of  linage  data* 

imax  Humber  of  frequencies. 

imi  Single  array  containing  image  height  data, 

jmax  Humber  Of  directions* 


Temporary  Variables 


iaddr  Index  or  address  o£  the  first  value  of  a  given  integer  height* 
iv  Integer  height  value, 

m,  n  Co  loop  indices* 

numv  Humber  of  occurrences  of  a  given  integer  value. 


Output  Variables 


ind  Array  containing  indices  of  inti  for  the  sorted  array, 

iorder  iorder(i)  is  the  index  of  imi(i)  in  the  sorted  array* 


parameter  (imax  ~  24,  jmax  =  23,  nmax  “  imax  *  jmax,  ihmax  -  100) 
dimension  imi(nmax),  numv  (ihmax)  P  iaddr  (ihmax) 
dimension  iorder(nmax) r  ind (ihmax) 


c 

e  Count  the  number  of  occurrences  of  each  integer  height. 

§£ 


numv  “  0 
do  n  -  1,  nmax 
iv  -  imi (n) 

numv ( iv )  -  numv { iv )  +  X 
end  do 


c  Compute  the  starting  address  of  each  integer  height* 

c 

iaddr (1)  =  1 

da  m  s  1,  ihmax  -  1 

iaddr  (m  t  1)  =  iaddr (m)  +  numv (m) 
end  do 


e  Compute  the  new  order  of  the  points, 

•d 

da  n  *  1,  umax 
iv  -  imi ( n ) 
larder (n)  =  iaddr (iv) 
iaddr (iv)  -  iaddr ( iv)  +  1 
end  do 

is? 

c  Sort  the  points* 

c 

do  n  ==  1,  nmax 

ind (i order (n) )  -  n 
end  do 


Figure  5.  Fast  sort  algorithm 
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the  radix  sort  invented  by  Harold  H.  Seward  at  Massa¬ 
chusetts  Institute  of  Technology  in  1954.  The  remainder 
of  this  article  explains  in  detail  the  fast  sort  algorithm  as 
it  can  potentially  help  any  user  who  needs  a  sort 
routine.  To  illustrate  the  steps,  consider  the  small 
example  shown  in  Figure  6  where  the  heights  of  the 
reciprocals  of  the  energy  density  image  data  have  been 
rounded  to  integers  between  1  and  5.  The  matrix  has 
four  frequency  values  and  four  directions.  The  four 


steps  for  the  fast  sort  are  done  with  four  simple  loops 
that  will  now  be  described.  Figures  6  and  7  show  the 
intermediate  values  after  the  steps.  Figure  6a  shows 
that  the  data  are  stored  in  the  single  array  row-wise, 
Figure  6b,  the  imager  image  data,  Figure  6c,  the 
i order  array,  and  Figure  6d,  the  watershed  partition¬ 
ing  with  the  highest  points  represented  by  the 
symbol.  As  in  the  continental  divide  example,  two 
watersheds  exist. 
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Figure  6.  Small  image  data  example 
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Figure  7.  Intermediate  data  for  the  sample  problem 


Count  the  Number  of  Occurrences 
of  Each  Integer  Height 

numv  =  0 
do  n  =  1,  nmax 
iv  =  imi (n) 

numv ( iv)  =  numv ( iv)  +  1 
end  do 

Here  the  array  numv  that  contains  the  number  of 
occurrences  of  a  given  value  is  first  initialized  to  zero. 
Next,  each  time  a  particular  value  is  encountered  while 
going  through  the  entire  list  of  nmax  (16  in  the  small 
example)  points,  the  number  of  occurrences  of  that 
value  is  incremented  by  one.  As  shown  in  Figure  7,  the 
small  test  problem  in  Figure  6  has  three  1 S,  six  2’s, 
three  3’s,  no  4’s,  and  four  5’s. 


Compute  the  Starting  Address 
of  Each  Integer  Height 

iaddr(l)  =  1 

do  m  =  1  ,  ihmax  -  1 

iaddr(m  +  1)  =  iaddr(m)  +  numv(m) 
end  do 

Here,  the  array  iaddr  contains  where  each  given 
integer  height  starts  in  the  sorted  array.  Starting  with 
an  address  of  1  for  the  lowest  height  of  1 ,  the  starting 
address  of  2  is  simply  1  plus  the  number  of  1  ’s;  the 
starting  address  of  3  is  the  starting  address  of  2  plus 
the  number  of  2’  s;  etc.  When  there  are  no  occurrences 
in  a  given  integer  height,  the  starting  address  is  mean¬ 
ingless.  It  does  not  interfere  with  the  working  of  the 
algorithm,  however. 
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Compute  the  New  Order  of  the  Points 

do  n  =  1,  nmax 
iv  =  imi (n) 
iorder(n)  =  iaddr(iv) 
iaddr(iv)  =  iaddr(iv)  +  1 
end  do 

The  array  i order  is  similar  to  telling  where  a  given 
horse  has  placed  in  a  horse  race.  For  instance,  the  first 
1  is  first  in  the  sorted  array  of  the  sample  data  set,  and 
the  last  5  has  the  last  or  sixteenth  place.  For  each  data 
point,  the  above  loop  determines  the  current  starting 
address  of  its  iv  value  in  the  sorted  array  and  gives 
the  respective  data  point’s  iorder  value  that  address. 
The  starting  address  of  the  given  iv  value  is  then 
incremented  by  1  so  that  the  next  time  this  iv  value  is 
encountered,  its  place  is  one  more  than  the  previous 
occurrence.  To  illustrate,  the  first  1  is  in  first  place,  the 
second  1  is  in  second  place,  and  the  third  1  is  in  third 
place  in  the  ordered  array  for  the  sample  problem. 

Sort  the  Points 

do  n  =  1,  nmax 

ind (iorder (n) )  =  n 
end  do 

Finally,  the  array  ind  gives  a  list  of  the  indices  pointing 
to  the  original  imi  array  that  can  now  be  used  to 
create  the  sorted  array  given  in  Figure  7.  That  is, 
imi  ( ind  (n)  )  is  the  nth  value  of  the  sorted  array . 

Future  Work 

The  capability  shown  in  Figure  4  and  discussed  above 
will  soon  be  available  in  the  latest  version  of 
WAVEWATCH  III. 
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Using  the  User  Interface  Toolkit  (UIT)  to  Run  Combat  Models 

By  Dr.  Philip  Amburn,  James  A.  Moyers,  and  Dr.  David  R.  Pratt,  Aeronautical  Systems  Center  Major 
Shared  Resource  Center,  Wright-Patterson  Air  Force  Base,  Ohio 


Forces  Modeling  and  Simulation  (FMS)  is  one  of  the 
10  Computational  Technology  Areas  (CTAs)  of  the 
User  Productivity  Enhancement  and  Technology 
Transfer  (PET)  program  of  the  HPCMP .  Dr.  David 
Pratt  is  the  FMS  Functional  Area  Point  of  Contact  and 
conceived  and  initiated  the  Divide  and  Conquer  project. 
Jamie  Moyers  and  Dr.  Philip  Amburn  are  the  FMS 
team  that  worked  on  the  design  and  implementation  of 
Divide  and  Conquer  and  modified  Divide  and  Conquer 
to  use  the  User  Interface  Toolkit.  The  overall  FMS 
strategic  objectives  are 

>  Leverage  the  HPC  experience  and  techniques 
within  other  CTAs  to  improve  FMS  support  to 
the  operational  warfighter. 

>  Identify  new  technologies  appropriate  for  model¬ 
ing  and  simulation  of  military  forces  in  HPC 
environments. 

>  Transition  the  technologies  to  operational  military 
programs. 

Military  analysts  are  potential  FMS  users  of  HPC 
resources.  These  analysts  are  primarily  interested  in 
codes  that  model  complete  things  and  are  focused  on 
the  effects  of  tradeoffs  and  employments  that  affect 
tactical  and  strategic  outcomes.  Figure  1  displays  the 
Modeling  and  Simulation  pyramid  of  U.SAir  Force 
(USAF)  analytic  models  with  several  combat  models 
organized  into  four  categories.  The  PET  program  has 
funded  work  for  over  2  years  on  a  project  called  Divide 
and  Conquer  (DAC),  which  is  a  tool  to  increase  the 
productivity  of  military  analysts  as  they  run  combat 
models  using  HPC  resources. 

Divide  and  Conquer 

DAC  was  designed  to  help  military  analysts  simulta¬ 
neously  submit  multiple  runs  of  a  combat  model,  where 
each  run  has  a  different  set  of  input  parameters,  often 
called  a  parameter  sweep.  These  co  mbat  models 
frequently  require  many  input  files,  and  some  models 
require  100  different  files;  but  for  a  particular  study  an 
analyst  will  routinely  need  to  vary  only  a  subset  of  all 
the  parameters  in  these  files.  The  goals  of  DAC  are 

>  Reduce  the  wall  clock  time  it  takes  to  conduct  an 
analysis 

>  Provide  more  complex  analyses  in  shorter  time 

>  Provide  a  scalable  environment 


Figure  1.  USAF  Analytic  Model  Hierarchy 


And  the  constraints  are 

>  Don’t  touch  my  model! 

>  I  want  to  work  at  my  desk! 

To  meet  these  goals,  we  developed  a  Web-based 
interface  that  collects  information  for  the  model  runs 
and  then  submits  the  HPC  jobs  for  the  user.  Figure  2 
shows  one  of  the  Web  pages  from  DAC  that  ana  lysts 
can  use  to  set  the  parameter  values  for  a  model  run.  At 
the  top  of  the  page  is  a  list  of  three  input  files  in  which 
these  parameters  will  be  varied.  There  are  eight 
parameters  to  be  varied;  some  of  the  parameters  are 
floating  point  numbers,  while  others  are  integers;  some 
have  a  list  of  values,  while  others  will  be  varied  like  a 
“for”  loop.  DAC  submits  an  individual  HPC  job  for 
each  combination  of  parameter  values. 

Air  Force  Institute  of  Technology  Use 
of  Divide  and  Conquer 

Early  users  of  DAC  are  Dr.  J.  O.  Miller  and  his 
students  at  the  Air  Force  Institute  of  Technology 
(AFIT).  Dr.  Miller  is  teaching  classes  and  conducting 
ongoing  research  in  combat  modeling  atAFIT,  and  they 
use  several  of  the  constructive  models  on  the  AF 
Studies  and  Analysis  list.  He  is  interested  in  using  HPC 
resources  to  support  both  his  research  and  classroom 
activities.  Dr.  Miller  has  the  following  comments  about 
DAC. 
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Figure  2.  Divide  and  Conquer  Web  page  for  modifying  parameters 


“What  makes  DAC  good  for  us  is  it  allows  our 
students,  primarily  analysts  and  not  hard  core 
computer  types,  to  be  able  to  harness  the  power 
of  a  ‘supercomputer’  with  a  combat  model 
without  having  to  worry  about  UNIX  or  even  the 
specific  procedures  required  to  get  one  of  these 
difficult  to  execute,  data  intensive,  combat 
models  to  run.  Not  only  can  they  execute  these 
models  with  this  nice  web-based  interface,  but 
they  can  also  set  up  a  sophisticated  experimental 
design  (given  that  we  have  identified  the  factors 
we  want  to  change  and  how  we  want  to  vary  the 
levels)  to  easily  explore  a  potentially  large 
solution  space.  We  have  used  DAC  in  both  my 
Joint  Combat  Modeling  I  and  II  classes  (first 
focuses  on  high  resolution  models  -  engagement 
and  mission  level,  while  the  second  is  aio 
referred  to  as  Aggregated  Combat  Modeling, 
putting  us  at  the  campaign  level).” 


Incorporating  UIT  within  DAC 

The  DAC  system  is  written  in  Python  and  uses 
TurboGears  for  database  and  Web  hosting.  The  UIT 
provides  the  capabilities  needed  to  connect  to  HPC 
resources  from  outside  the  HPCMP  environment. 

DAC  runs  on  a  Web  server  and  needs  to  authenticate 
users,  copy  files  to  and  from  HPC  machines,  etc.  The 
UIT  API  provides  easy  access  to  HPC  resources,  and 
it  has  been  approved  by  the  HPCMP  security  office — 
two  significant  benefits.  To  make  this  as  easy  to  use 
with  DAC  as  possible,  we  wrapped  UIT  API  calls  with 
two  Python  classes,  UITConnectionFactory  and 
UITConnection.  UITConnectionFactory  supports 
Kerberos  authentication  and  establishes  a  connection  to 
an  HPC  host.  UITConnection  wraps  a  subset  of  the 
UIT  API  for  file  transfer,  file  and  directory  manage¬ 
ment,  etc.  By  wrapping  the  UIT  API  calls  within  these 
Python  classes,  we  insulated  the  DAC  code  from  the 
Python  SOAP  interface  requirements  and  details  of  the 
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UIT  API.  For  example,  the  following  four  lines  of  code  authenticate  a  user  and  copy  a  file  from  the  local  host  (a 
Web  server  for  DAC)  to  the  HPC  machine.  The  second  line  authenticates  a  user;  the  third  line  establishes  a 
connection  to  hpcl  1  .asc.hpc.mil;  and  the  fourth  line  copies  a  file  from  the  HPC  to  the  local  host. 

uitcf  =  UITConnectionFactory  ( ) 

uitcf . authenticate (username ,  password,  secureid_code ,  "  ASC . HPC . MIL" ) 

hpcll  =  uitcf . getConnection ( "hpcll" ) 

hpcll . copyFileFromHPC ( "/haf sll/moyers j /f oo . dat" ,  "/tmp/f oo . dat" ) 

The  Python  code  that  wraps  the  authenticate  method  in  the  UITConnectionFactory  class  is  below  and  shows  how 
this  encapsulation  insulates  the  calling  program  from  details  of  the  UITAPI. 

# 

# - 

#  authenticate:  get  Kerberos  credentials 

# 

def  authenticate (  self,  name,  password,  passcode,  realm  ): 
credentials  =  """ 

<credentials> 

<username>%s@%s< /username > 

<password>%s</password> 

<passcode>%s< /pass code > 

</credentials>" " "  %  (name,  realm,  password,  passcode) 

self . userName  =  name 


try : 

self . authtoken  =  self . server . authenticateUser (credentials , 

' MSRC_KERBEROS ' ) 

except : 

status  =  False 

print  "\n — >  failed  to  get  kerberos  credentials\n" 

else : 

status  =  True 

#print  "\n — >  got  credentials\n" 

return  status 

Summary 

Divide  and  Conquer  (DAC)  now  requires  the  UIT,  and  wrapping  a  subset  of  the  UIT  API  in  two  Python  classes 
made  the  switch  to  using  the  UIT  for  HPC  interaction  “easy.”  The  effort  to  learn  the  UIT  and  build  and  test  the 
Python  classes  required  only  4  weeks  of  our  time.  While  it  is  still  early  in  this  effort,  the  approach  has  proven  quite 
useful  and  effective.  DAC  provides  a  capability  that  FMS  presents  to  various  military  analysis  organizations,  and  it 
is  critical  to  offer  them  this  capability  for  easy  use  of  HPC  resources  to  run  standard,  approved  combat  models. 
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What  is  ezVIZ? 

By  Randall  Hand 


In  2000,  the  top  computer  in  the  world,  IBM’s  ASCI 
White,  had  a  maximal  achieved  performance  of  2.38 
TFLOPS.  In  2004,  77  computers  were  more  powerful 
than  ASCI  White,  which  is  now  30  times  slower  than 
the  fastest  computer.  This  trend  is  expected  to  con¬ 
tinue,  with  the  first  PFLOPS  machine  being  available  in 
2007.  With  the  ubiquitous  availability  of  TFLOPS 
machines,  the  associated  data  are  growing  at  an 
increasing  rate.  Terabyte  data  sets  are  now  common. 

Gaining  useful  information  from  the  data  is  a  challeng¬ 
ing  task.  This  task  often  falls  to  the  researcher  or  a 
visualization  scientist.  Extracting  insight  from  a 
multiterabyte  data  set  presents  the  researcher  with 
several  problems.  These  problems  include  transfer  and 
storage  of  the  data,  graphics  hardware  to  visualize  it,  as 
well  as  having  visualization  software  capable  of  han¬ 
dling  the  data. 


ezVIZ  tackles  the  visualization  problems  of  the  researcher 
by  providing  one  of  two  mechanisms.  The  first  is  a  batch 
visualization  capability.  This  batch  capability  allows  the 
users  to  create  images  from  their  data  while  the  data  still 
reside  on  the  supercomputer.  These  images,  which  are 
less  than  a  few  megabytes  in  size,  can  then  be  moved 
with  ease  to  the  researcher ’s  workstation.  Storage  and 
network  bandwidth  are  no  longer  a  concern  when 
visualizing  the  data.  The  second  mechanism  is  to  provide 
a  Web  interface  to  visualizing  the  data.  This  mechanism  is 
currently  under  development. 

Current  State  of  ezVIZ 

ezVIZ  is  now  officially  released  and  available  on  all 
HPC  systems  at  ERDC.  Currently  at  version  1.1,  it  has 
improved  significantly  from  the  early  beta  tests  and 
boasts  a  wide  feature  set  including  support  for  over 
30  input  formats  and  7  rendering  operations. 
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In  addition  to  the  previous  rendering  operations,  there 
are  now  a  few  new  rendering  modes: 

1 .  Model/Surface  -  Render  polygonal  geometry  or 

extract  outer  volume  surfaces 

2.  Flow  Glyphs  -  Place  objects  in  a  flow  field, 

oriented  and  scaled  by  the  flow 

3.  Streamlines  -  Generate  lines  to  represent  the  flow 

4.  Volume  Rendering  -  Full  ray-cast  visualizations  of 

3-D  volumes 

5.  Isosurface/Contour  -  Extract  a  polygonal  surface 

or  contour  line  from  a  volume 

6.  NEW!  Cross  Section  -  Extract  a  planar  cross 

section  from  a  data  set 

7.  NEW!  Molecule  -  Generate  ball-stick  renderings 

from  molecular  data 


PNG,  BMP, 
JPG,  TIF, 

PNM 

BYU 

DEM 

VTK  XML  & 
Legacy 

Metalmage 

Wavefront  OBJ 

Stanford  PLY 

STL 

Unigraphics 

Facet 

Protein  Data 
Bank 

XYZ  XMol  Files 

ASCII  Facet 

Gaussian 

Cube 

CFDViz’s  Poly 
Format 

POP  Ocean  files 

AVS  UCD 

Plot3D 

Xdmf/Xmdf 

IRC  Info 

Packet 

Exodusll 

CTH  (SpyPlot) 

Ensight 

Quoddy 

ADCIRC 

Input  File  Formats 


Home 

HowTos 

ExampleScripts 

Documentation 


close  close  others  view  permalink  references  jump 

ezVIZ  Script  Generator 


Input 


Input  Format:  I  Automatic  Selected  based  on  file  extension  _^J 
-  Input  Options - 


File:  f 


-  Regioning - 
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by  semicolons  The  result  of  all  formula  will  be  double- 
precision  floating  point  and  the  number  of  components 
will  be  automatically  detected  by  the  functions  used 
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Documentation  for  all  of  these  features  can  be  found 
on  the  new  ezVIZ  Web  site.  Simply  point  your  browser 
to  https ://solutionhpc . erdc.hpc.mil/EZHPCWebClient/ 
ezVIZ/ez Viz.html,  or  click  on  the  ezVIZ  link  from 
ezHPC.  On  the  ezVIZ  Web  site,  you  can  find  full 
documentation  of  both  ezVizGeneric  and  the  ezVIZ 
C++  API,  along  with  examples  and  “how  to’s”  for  all 
major  features. 

What’s  in  store? 

There’s  still  much  active  development  going  on  for 
ezVIZ.  In  the  near  future,  you  can  expect  to  see  a 
Web-based  ezVIZ  Script  Generator  to  help  you  build 
the  ezVizGeneric  scripts  and  submit  the  jobs.  That  Web 
site  will  offer  a  fully  interactive  and  dynamic  interface 
for  building  scripts  that  will  result  in  a  fully  usable  script 
to  submit  to  the  HPCs,  or  even  a  simple  “Submit” 
button  to  submit  the  render  job  via  ezHPC. 

Also,  the  next  version  (vl.2)  of  ezVIZ  is  already  under 
development.  Expected  features  include  the  following  : 

>  Support  for  Hierarchical  and  AMR  data  sets 

>  Better  use  of  Parallel  Resources  -  For  better  use 
on  clusters  and  the  XT3 

>  Plugin  support  for  Input  Formats  -  If  ezVIZ 
doesn’t  read  your  data  natively,  write  your  own 
reader 

>  Combination  Visualization  -  Combine  the  results 
of  multiple  ezVIZ  scenes  into  a  single  image 

>  More  input  formats 

>  Much  more... 


ezVIZ  already  boasts  over  52,000  executions  and  5,000 
CPU  hours  in  the  first  6  months  of  logging. With  that 
data  we’ve  learned  a  lot  about  how  visualization  isused 
on  the  systems  and  what  kind  of  functionality  is  mismg. 
We’ve  taken  huge  steps  to  filling  in  these  gaps,  and  as 
a  result  have  seen  significant  improvements  in  stability 
features,  and  ease  of  use.  Only  through  continued 
usage  and  development  can  ezVIZ  continue  to  be 
useful  and  provide  the  constantly  changing  capabilities 
that  have  proven  useful. 


[input] 
f ormat=VTK 

[output] 
f ormat=PNG 
size=HDTV 
f ile=f rame .png 

[render] 

operation=isosurface 
scalarf ield= fraction 
isovalue=0 . 5 
elevation=-88 
zoom=12 . 0 

[colormap] 

stretch=on 

colorf ield=Velocity 

colorcomponent=magnitude 

Example  ezVizGeneric  Script 


Example  ezVIZ  output 
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Improving  User  Experience  and  Reducing  Time  to  Solution 

By  Jay  Cliburn 


The  ERDC  MSRC  has  major  improvements  planned 
for  the  network  and  mass  storage  infrastructure  in  the 
coming  months.  First,  we  plan  to  replace  the  existing  1- 
gigabit  Ethernet  backbone  with  a  10-gigabit  Ethernet 
backbone,  which  will  dramatically  improve  data  trans¬ 
fer  speed  between  HPC  syst  ems  and  the  mass  storage 
Data  Management  System  (DMS).  In  conjunction  with 
this  change,  we  also  plan  to  increase  the  size  of  the 
Fibre  Channel  disk  cache  on  the  DMS  from  about  6 
terabytes  to  about  25  terabytes,  quadrupling  the  amount 
of  data  the  DMS  can  hold  before  flushing  files  to 
magnetic  tape. 


The  primary  goal  of  these  changes  is  to  reduce  the  time 
it  takes  to  push  or  pull  data  files  between  HPC  systems 
and  the  mass  storage  system.  Our  continuing 
overarching  motivation  is  to  improve  overall  user 
experience  and  to  reduce  time  to  solution.  These 
infrastructure  upgrades  take  significant  strides  in  that 
direction. 

If  you  have  questions  or  need  additional  information, 
contact  the  HPC  Service  Center  at 
msrchelp@erdc.hpc.mil. 


FOUNDRY  NETWORKS 
RX-S  PROVIDES  A 
NEW  10  GIGABIT  BACKBONE 


SUN  FIRE  15K  & 
25  TERABYTES 
OF  USABLE 
SAMQFS  DISK 
CACHE 


SGI  03K  HPC 
HOST 


Cray  XT3  HPC 
HOST 
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A  Purpose  Driven  ...  Web  Site? 

By  Glen  Browning ,  ERDC  MSRC 


In  January  of  2004,  in  response  to  feedback  from  many 
of  our  users,  we  at  the  ERDC  MSRC  undertook  a  bold 
initiative  to  improve  the  way  we  communicate  with  our 
users.  We  began  with  five  RSS  newsfeeds,  a  couple  of 
tin  cans,  and  a  whisper,  “Psst!  We’ve  got  a  better  way 
to  communicate.” 

Two  and  a  half  years  later,  this  simple  idea  has  grown 
into  a  contagious  philosophy  that  continues  to  spread 
within  our  organization,  affecting  virtually  every  activity 
of  User  Services.  Perhaps  nowhere  is  this  change 
more  clearly  seen  than  in  our  new  Web  site  — 
www.erdc.hpc.mil.  As  you  might  expect,  the  cumber¬ 
some  frames  and  the  tired  mid- 6 90s  design  have  been 
replaced  with  a  unified  page  layout  and  a  fresh  new 
look,  but  the  most  important  change. . .  the  change  that 
makes  all  the  difference. . .  is  not  a  change  of  design, 
but  a  change  of  purpose. 

This  is  more  than  a  clever  attempt  to  sound  trendy.  In 
the  past  the  purpose  of  our  Web  site  was  vague  and 
constantly  changing,  making  meaningful  content  an 
elusive  target.  To  solve  this  problem,  we  decided  early 
in  the  redesign  process  to  focus  on  a  single,  well- 
defined  purpose  —  to  be  useful  to  our  users.  Then  we 
did  something  truly  outrageous.  We  picked  up  the 
phone  and  started  calling  them.  Who  better  to  tell  us 
what  users  need?  We  asked  them  what  they  wanted 
from  our  Web  site.  What  could  we  do  that  would  make 
their  lives  easier?  What  would  be  useful?  And  then  we 
listened,  and  the  response  was  eye-opening. 

“...  the  most  important  change... 

is  not  a  change  of  design, 
but  a  change  of  purpose.” 

We  learned  that  many  of  our  users  were  new  to  high 
performance  computing  and  had  never  worked  in  a 
UNIX  environment  before.  For  many  of  these  users, 
discussions  about  Kerberized  systems  and  queue 
priorities  seemed  like  a  foreign  language.  Faced  with  a 
mountain  of  new  concepts  and  terminology  they  just 
needed  more  help  getting  started,  and  many  didn’t 
know  where  to  turn.  A  lot  of  users  found  the  old  site 
confusing  and  often  got  lost  trying  to  find  information. 
We  also  learned  that  the  stiff,  formal  language  of  the 
site  was  daunting  to  many  and  that  the  content  was 
often  too  dated  to  be  useful.  And  finally,  we  learned 


We've  got  a  better  way 

to  communicate. 


Postcards  sent  to  every  ERDC  MSRC  user  announced 
the  debut  of  our  RSS  newsfeeds 

that  many  people  just  wanted  to  know  if  a  system  was 
up  or  down.  With  these  things  in  mind,  we  began  laying 
out  the  blueprint  for  a  site  that  would  truly  meet  these 
needs  and  many  more. 

Our  first  step  was  to  eliminate  everything  that  wasn’t 
inherently  useful.  In  some  cases  this  meant  getting  rid 
of  content  altogether.  In  others,  it  meant  rewriting 
documents  from  scratch  or  merging  multiple,  incom¬ 
plete  documents  to  create  a  single  useful  page. 
Throughout  this  process  nothing  was  sacred,  and  whole 
sections  of  the  original  Web  site  were  retired  simply 
because  they  weren’t  useful.  The  result  was  a  much 
leaner  body  of  content  with  far  fewer  pages  and  useful 
information  on  virtually  every  page. 

But,  we  didn’t  stop  at  the  level  of  sections  and  pages. 

In  many  documents,  the  wording  of  every  sentence 
was  scrutinized.  The  formal,  third  person  style  of 
writing  was  replaced  with  simpler  more  conversational 
language.  We  kept  formality  to  a  minimum,  and  clarity 
and  brevity  became  the  rule  of  thumb.  We  made  it  our 
goal  to  never  use  20  words  to  say  what  could  be  said 
more  clearly  in  10. 

We  then  turned  our  attention  to  the  structure  of  the  site. 
After  all,  it  doesn’t  matter  how  useful  and  clear  infor¬ 
mation  is  if  people  can’t  find  it  when  they  need  it. 
Starting  from  scratch,  we  built  a  new  site  structure 
around  the  information  that  users  most  frequently 
needed  . . .  system  news,  documentation,  hardware  and 
software  details,  services,  and  local  information.  We 
made  every  effort  to  create  an  intuitive  structure  in 
which  information  could  be  easily  located.  In  addition 
to  traditional  navigation  menus,  we  added  shortcut 
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questions  linking  directlyto  important  information 
deeper  in  the  site.  To  the  System  News  section,  we 
added  a  system  status  summary  page  with  status  icons 
color-coded  in  the  familiar  traffic  light  motif.  Green  is 
up.  Yellow  means  possible  problems.  Red  is  down.  Itfe 
that  simple.  To  complete  the  package,  we  added  a 
search  engine  so  users  could  find  what  they  needed  as 
quickly  as  possible. 


Color-coded  icons 
identify  system  status 


Next,  we  shifted  focus  to 
the  needs  of  new  and 
prospective  users,  many 
of  whom  are  not  only 
new  to  our  Center  but  to 
high  performance  com¬ 
puting  as  well.  For  those 
wanting  to  become  users, 
we  added  step-by-step 
instructions  to  walk  them 
through  the  complicated 
process  of  getting  an 
account.  Following  this, 
we  created  a  guide  for 
new  users  to  take  them 
through  their  first  logon 
session  and  answer 
common  questions  about 
Kerberos  authentication, 
job  submission  and 
queues,  issues  unfamiliar 
to  users  who  have 
worked  mostly  with  PCs. 


A  detailed  set  of  pages  was  put  in  place  to  answer  the 
top  four  most  frequently  asked  questions,  and  the 
remaining  FAQs  were  divided  into  Beginner  and 
Advanced  documents  so  that  the  questions  could  be 
addressed  in  greater  detail.  The  Quick  Start  Guides  and 
User  Guides  were  enhanced  with  more  thorough 
explanations,  and  clearer,  more  complete  examples 
were  added.  Information  about  queue  limits  and  system 
utilities  were  reformatted  into  easy-to-read  tables,  and 
Tips  &  Tricks  documents  were  updated  and  grouped 
together  by  the  systems  to  which  they  applied.  Of 
course,  we  knew  that  we  would  never  be  able  to 
anticipate  every  user’s  question,  but  it  wouldn’t  be  for 
lack  of  trying. 

With  this  exhaustive  (and  exhausting)  array  of  en¬ 
hancements  in  place,  we  felt  that  we  finally  had  a  site 
that  would  serve  our  users  well.  Great  effort  had  gone 


“ ...  it  doesn’t  matter  how  useful  and  clear 
information  is  if  people  can’t  find  it. . . 


into  ensuring  that  every  known  issue  was  addressed, 
and  that  we  had  a  framework  in  place  that  would  allow 
the  site  to  continue  to  meet  the  needs  of  our  users  in 
the  future.  So  on  January  3,  2006,  we  launched  the  new 
ERDC  MSRC  Web  site  and  waited  to  see  how  our 
users  would  respond.  And  the  response  was. . .  well, 
see  for  yourself.  These  are  just  a  few  of  the  com¬ 
ments  we  received. 

. .  very,  very  good.  To  be  able  to  click  about 
three  buttons  and  see  the  timely  accurate,  and 
complete  status  of  each  system,  complete  with 
explanations,  estimated  uptimes,  and  color- 
coded  pictures,  is  worth  a  great  deal.” 

“...well  thought  out 
and  impressive...” 

"...  an  excellent  setup.” 

“I  love  the  new  site.  It§  ARTISTICALLY 
pleasing.  You  did  a  fabulous  job  with  the 
new  design.  What  an  inspiration...” 

“Awesome!!” 

The  enthusiastic  response  of  our  users  was  quite 
frankly  more  than  we  ever  expected,  and  it  confirmed 
for  us  that  we  were  indeed  on  the  right  track;  but  it  also 
reminded  us  of  how  much  more  could  be  done.  So  after 
a  brief  celebration  (and  some  much  needed  sleep)  we 
found  ourselves  asking  a  familiar  question,  “How  can 
we  make  this  better?  Where  do  we  go  from  here?”  Of 
course,  we  have  lots  of  ideas,  but  the  answer  will  be 
determined  by  the  needs  of  our  users.  So,  if  you  have  a 
suggestion. . . 

we  want  to  hear  from  you! 

Call  us  at  1-800-500-4722,  or  send  e-mail  to 
mwebmaster@erdc.hpc.mil.  Our  purpose  is  to  be 
useful  to  YOU,  so  tell  us  your  needs  and  see  what 
happens. 

Here  we  go  again!  I  think  I  may  just  put  a  cot  in  my 
office! 
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www.erdc.hpc.mil 


Major  Shared  Resource  Center 


Documentation 


ructnmpr  Service 


about  what  we  do’  Take  a 


Who  We  Are 

Contact  US  ^b.^aTnewsletter,  the  Resource 

planning  A  Visit 


The  Resource 

look  at  the  following  issues  of 


The  latest  and  greatest 


. 

c  ,\l  2003 


Looking  for  past  issues  of  the  Resource?  They’re  all  available  online  at 
http:/ /www.  erdc.  hpc.  mil/aboutUs/The_Resource/home 
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New  ERDC  MSRC  Acting  Director 

by  Rose  J.  Dykes 


Greg  K.  Rottman  was  named  the  ERDC  MSRC  Acting 
Director  on  September  3  when  John  E.  West  became 
the  ERDC  ITL  Acting  Deputy  Director  for  the  remain¬ 
der  of  2006.  Just  previous  to  this  appointment,  he  was 
serving  as  an  ERDC  MSRC  Assistant  Director  with 
responsibility  for  all  business  aspects  of  the  Center  In 
2005  he  received  the  HPCMP  Hero  Award  for  the 
category  Innovative  Management  for  demonstrating 
creative  business  practices  to  improve  overall  HPCMP 
business  model. 

The  first  DoD  HPCMP  Metacomputing  Grid  Workshop 
was  hosted  by  the  ERDC  MSRC  in  early  2002.  Attend¬ 
ees  from  across  the  Nation  attended  this  4-day  event 
for  which  Greg  served  as  workshop  coordinator.  Greg 
had  served  as  the  HPCMP  Project  Manager  for  the 
metacomputing  initiative,  developing  proposals,  sched¬ 
ules,  and  budgets  for  research  work  to  be  done.  His 
responsibility  included  coordination  of  project  activities 
with  a  geographically  dispersed  team  and  performing 
liaison  functions  with  working  groups  dedicated  to 
metacomputing  technologies. 

In  2001,  Greg  completed  the  Army’s  3-year  premier 
leadership  development  program  for  engineers  and 
scientists — one  of  only  two  people  from  ERDC  with 
this  accomplishment.  He  was  also  serving  as  the 
Government  lead  for  HPC  systems  operation  at  the 
ERDC  MSRC  at  this  time. 

Greg  came  back  to  ERDC  in  2000  as  a  civil  engineer 
working  for  the  ERDC  MSRC.  Just  previous  to  joining 
the  ERDC  MSRC,  Greg  spent  5  years  in  the  Programs 
and  Project  Management  Division  at  the  U.S.  Army 
Engineer  District,  Vicksburg.  He  had  returned  to 
Government  service  at  the  ERDC  ITL,  where  he 
worked  in  Customer  Assistance  and  served  as  project 
engineer  and  project  manager  for  the  Electronic 
Information  Exchange  Project,  before  the  Vicksburg 
District  assignment. 

In  1984  Greg  went  to  work  for  Tektronix,  Inc.,  as  a 
system  analyst,  advancing  through  the  ranks  of  senior 


Multitasking,  fun-loving  Greg  (far  right)  was  a  quartet 
member  of  the  “ERDC  MSRC  Singing  TERROR  flops ” 
at  the  MSRC  “We  Are  Watching  You”  surprise 
celebration  in  honor  of  John  West.  Also  pictured 
(left  to  right)  David  Stinson  on  keyboard;  John  Mason, 
guitar;  Trent  Townsend,  guitar;  and  otherquartet 
members  “Tiny”  Tim  Dunaway,  Dr.  Jeff  Hensley 
and  Mickey  Robertson 

systems  analyst,  product  support  analyst,  national  sales 
training  manager,  and  sales  engineer.  Previous  to  this 
he  had  worked  as  a  hydraulic  engineer  for  the  U.S. 
Army  Corps  of  Engineers  in  Fort  Worth,  Texas,  after 
earning  his  B.S.  degree  from  Texas  Tech  University  in 
1977. 

In  addition  to  being  honored  with  three  Commandefs 
Awards  during  the  last  few  years,  Greg  has  also  se  rved 
as  a  member  of  the  Diocese  of  Jackson  School  Board, 
Jackson,  Mississippi,  and  member  of  the  Development 
Committee  at  the  Vicksburg  Catholic  Schools, 
Vicksburg,  Mississippi.  Under  his  leadership  as  the 
Grand  Knight,  the  Vicksburg  Knights  of  Columbus 
received  the  highest  award  in  the  State  of  Mississippi 
for  the  2  years  Greg  served. 

The  ERDC  MSRC  feels  fortunate  to  have  Greg  as  a 
member  of  its  team  and  to  have  him  serve  as  its  Acting 
Director  while  John  West  is  serving  in  another  capacity. 
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About  PET 

By  Dr.  Jim  Pennington 

The  User  Productivity  Enhancement  and  Technology 
Transfer  (PET)  program  is  responsible  for  gathering 
and  deploying  the  best  ideas,  algorithms,  and  software 
tools  emerging  from  the  national  high  performance 
computing  infrastructure  into  the  DoD  user  community 
PET,  in  its  sixth  year  under  the  current  contract  with 
DoD’s  High  Performance  Computing  Modernization 
Program  (HPCMP),  is  supported  by  two  contracts:  the 
first  with  the  MOS  consortium  led  by  Mississippi  State 
University  and  the  Ohio  Supercomputer  Center;  the 
second  with  High  Performance  Computing  Technolo¬ 
gies  Inc.  (HPTi).  MOS  is  responsible  for  PET  Compo¬ 
nents  2  and  3,  which  are  hosted  by  the  Major  Shared 
Resource  Centers  (MSRCs)  at  the  Aeronautical 
Systems  Center  (ASC),  Wright-Patterson  Air  Force 
Base,  at  Dayton,  Ohio,  and  the  U.S.  Army  Engineer 
Research  and  Development  Center  (ERDC)  at 
Vicksburg,  Mississippi,  respectively  HPTi  is  responsible 
for  PET  Component  4,  which  is  hosted  at  the  Army 
Research  Laboratory  at  Aberdeen  Proving  Ground, 
Maryland.  The  functions  and  responsibilities  of  PET 
Component  1,  which  had  been  hosted  by  the  Naval 
Oceanographic  Office  (NAVO)  at  Stennis  Space 
Center,  Mississippi,  were  transferred  into  Components 
2  and  3  in  June  2006. 

The  PET  team  consists  of  university  and  industry 
partners  that  are  located  from  Hawaii  to  the  East 
Coast.  PET  brings  the  expertise  of  these  institutions  to 
bear  in  support  of  DoD  HPC  users  through  technology 
transfer,  training,  and  collaborative  assistance  in  several 
computational  technology  areas  (CTAs).  Functional 
area  points-of-contact  (FAPOCs)  are  provided  for 
each  of  the  CTAs  and  crosscutting  support  areas.  The 
FAPOCs  field  a  team  of  onsite  staff  located  at  the 
DoD  centers  with  HPC  user  concentrations  and  are 
supported  by  other  staff  at  the  PET  team  home  institu¬ 
tions.  The  PET  onsite  staff  supports  DoD  users  across 
the  HPCMP  and  not  just  the  users  at  their  respective 
locations.  PET  onsite  resources  at  the  ERDC  MSRC 
are  Dr.  Nathan  Prewitt  for  Computational  Fluid  Dy¬ 
namics  (CFD),  Dr.  Rick  Weed  for  Computational 
Structural  Mechanics  (CSM),  and  Dr.  Jeff  Hensley  for 
Environmental  Quality  Modeling  and  Simulation 
(EQM).  Drs.  Prewitt,  Weed,  and  Hensley  are  available 
to  assist  all  HPC  users  in  DoD  just  as  assistance  is 
available  to  ERDC  users  in  other  CTAs  from  PET 
university  faculty  and  onsite  leads  at  other  MSRCs  and 
DoD  facilities. 


The  CTAs  are  operated  by  assigning  the  technical 
areas  to  one  of  the  three  components  with  each 
component  managed  by  a  Component  Point-of-Contact 
(CPOC).  Component  3  is  managed  by  Dr.  Jim 
Pennington,  the  CPOC.  Government  oversight  for  this 
component  is  provided  by  Mr  Chris  Merrill,  ERDC/ 
ITL,  who  serves  the  HPCMP  Office  as  the  PET 
Component  3  Technical  Assistant  (see  insert  on 
page  29).  PET  Component  3  is  responsible  for  collabo¬ 
rative  user  support  in  four  CTAs:  CFD,  CSM,  EQM, 
and  Climate/Weather/Ocean  Modeling  and  Simulation 
(CWO).  Component  3  is  also  responsible  for  one  of  the 
crosscutting  functional  areas,  Education,  Outreach,  and 
Training  Coordination  (EOTC). 

PET  Summer  Interns 

One  of  the  methods  to  influence  growth  of  the  DoD- 
speciflc  workforce  is  through  the  PET  Summer  Intern 
program,  which  is  coordinated  by  the  EOTC  crosscut¬ 
ting  functional  area.  The  intern  program  funds  students 
to  work  and  learn  at  a  DoD  HPC  Center  in  collabora¬ 
tion  with  a  PET  mentor.  The  students  see  firsthand 
what  opportunities  exist  in  the  DoD  centers  and  how 
exciting  the  world  of  HPC  can  be.  The  hope  is  that 
some  of  them  will  seriously  consider  a  career  at  a  DoD 
site. 

The  Summer  Intern  program  is  open  to  all  undergradu¬ 
ate  and  graduate  students  with  U.S.  citizenship.  Student 
recruitment  begins  in  the  winter  by  personal  contact 
with  university  deans,  department  heads,  senior  faculty 
and  other  points-of-contact.  A  mentor  will  direct  the 
student’s  summer  activities  with  the  PET  program.  The 
mentor  will  be  skilled  in  a  relevant  technical  area  and 
will  possess  the  needed  skills  in  interpersonal  relation¬ 
ships  to  articulate  appropriate  statements  of  work  and 
devise  individual  development  plans  that  are  consistent 
with  the  student’s  interests  and  capabilities.  The  mentor 
will  direct  the  student’s  day-to-day  activities  during  the 
summer  and  will  be  responsible  for  assigning  the 
student  meaningful  projects  that  are  not  only  of  value  to 
the  DoD  but  also  contribute  to  the  student’s  knowledge 
and  maturity  in  the  field  of  HPC. 

In  2006,  PET  sponsored  43  summer  interns,  8  of  whom 
spent  their  summer  at  the  ERDC  MSRC. 
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Corey  Bordelon  worked  under  the 
mentorship  of  John  Mason  in 
ERDC’s  Major  Shared  Resource 
Center  on  creating  a  new  system  to 
assist  in  the  retrieval  of  documents 
on  the  Online  Knowledge  Center 
site.  Corey  graduated  from  the 
University  of  Louisiana  at  Monroe  in  May  with  a  degree  in 
computer  science.  The  Marksville,  Louisiana,  native  is  attend¬ 
ing  the  University  of  Louisiana  at  Lafayette  for  his  master’s  in 
computer  science. 


/nos 


Naomi  Brown  worked  under  the 
mentorship  of  Drs.  Alvaro 
Lernandez  and  Paul  Bennett  in 
ERDC’s  Major  Shared  Resource 
Center.  While  working  with 
Lernandez,  she  learned  about 
meshless  methods  for  solving  model 
convection-diffusion  problems  and 
helped  extend  his  current  algorithm 
to  two-dimensional  problems.  Naomi’s  work  with  Bennett 
pertained  to  laser  propagation  through  nonlinear  materials.  She 
performed  a  series  of  tests  to  determine  where  the  code  needs 
to  be  more  robust.  Naomi  is  in  her  third  year  at  the  University 
of  Hawaii  at  Manoa  and  is  pursuing  a  bachelor’s  in  physics. 


Ihsan  Hakima  worked  in  ITL ’s 
Engineering  and  Information 
Systems  Division,  under  the 
mentorship  of  Dr.  Owen  Eslinger. 
They  were  developing  iterative 
methods  of  finding  the  thermal 
diffusivity  from  the  heat  equation  by 
the  finite  difference  method  and 
comparing  the  results  to  field  data.  Ihsan  is  anelectrical 
engineering  and  mechanical  engineering  major  at  the  Univer¬ 
sity  of  Alabama  at  Birmingham. 
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Caitlin  Wolfe  worked  under  the 
mentorship  of  Paul  Adams  in  the 
ERDC  Major  Shared  Resource 
Center’s  Scientific  Visualization 
Center  on  a  project  involving 
movement  in  water  to  create 
animation.  The  Brandon,  Missis¬ 
sippi,  native  is  majoring  in  aerospace  engineering  at  Mississippi 
State  University. 


Maryum  Haston  worked  under  the 
mentorship  of  Dr.  Nathan  Prewitt  in 
ERDC’s  Major  Shared  Resource 
Center,  trying  to  convert  a  visualiza¬ 
tion  code  to  run  under  Linux  and 
Windows  using  GTK+,  a 
multiplatform  toolkit  for  creating 
graphical  user  interfaces  (GUI), 
and  liblglade  to  produce  a  new  GUI  for  the  code.  The  Mobile, 
Alabama,  native  is  seeking  a  degree  in  mechanical  engineering 
at  the  University  of  Alabama  at  Birmingham. 
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GTK+  Examples  with  Open  GL 


•  Without  menu  bar  •  With  GTK+  menu  bar 


Steven  A.E.  Miller  is  working  on 
his  doctorate  in  aerospace 
engineering  in  computational  fluid 
dynamics  and  computational 
aeroacoustics  at  The  Pennsylvania 
State  University.  His  mentors  were 
Dr.  Thomas  C.  Oppe,  ERDC 
Major  Shared  Resource  Center 
and  Dr.  Zeki  Demirbilek,  CHL.  Seve,  a  native  of  Saline, 
Michigan,  is  working  on  direct  and  iterative  sparse  matrix 
solvers  for  CGWhve  and  updating  CGWive  on  various 
computer  platforms. 


Serial  Benchmarking  CGWave 


/HOS 


OSC 


1.E+06  1.E+07 

Log(»Eqn.  *  ffSpectral  Cmp.) 
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Kendria  Favors  worked  with 
Renee  Mullinax,  Engineering  and 
Informatic  Systems  Division.  The 
Atlanta,  Georgia,  native  worked 
on  implementing  enhancements  to 
the  PET  Online  Knowledge 
Center  using  JSP  communicating 
with  an  Oracle  Database. 

Kendria  attends  Albany  State  University  and  is  majoring  in 
mathematics  and  industrial  engineering  as  part  of  a  dual 
degree  program  with  the  Georgia  Institute  ofTechnology. 


yHQ5 


Step  3!:  Quarterly  Report  Options 

Fleue  note  that  the  contract  year  always  starts  in  June,  therefore  for  example  if  data  is  bemg  obtain  for  a  c< 
Course  Type  Regular  (In-Class)  v 
Quarter  1st  Quarter  June,  July.  Aug  ~ 

Year  2002  (YK2)  - 

Report  Type  Total  Number  of  Courses  v 
Sorted  by.  Functional  Area  » 

1  Submit  | 


Lan  Holloway ,  Clinton,  Mississippi, 
worked  under  the  mentorship  of 
Dr.  Sam  Cable,  ERDC  Major 
Shared  Resource  Center.  They 
looked  at  the  bow  shock  surround¬ 
ing  the  earth  and  magnetosphere 
and  how  the  density  velocity,  and 
pressure  of  the  wind  vary  in  this 
region.  Lan  is  a  sophomore  at 
Mississippi  State  University  and  is  majoring  in  mechanical 
engineering.  He  is  pursing  minors  in  mathematics  and  com¬ 
puter  science. 


KR  ytlOS 

Data  Examples 


s~\  s~\ 


(Text  and  photographs  of  Summer  Interns  courtesy  of  Marsha  Gay) 


Mr.  Chris  Merrill  is  the  new  PET  Component  3  Technical  Assistant.  This  compo¬ 
nent,  hosted  by  the  ERDC  MSRC,  includes  the  computational  technical  areas 
(CTAs)  of  Computational  Fluid  Dynamics  (CFD),  Computational  Sructural  Mechan¬ 
ics  (CSM),  and  the  newly  added  CTAs  of  Environmental  Quality  Modeling  and 
Simulation  (EQM),  and  ClimateA^ather/Ocean  Modeling  and  Simulation  (CWO). 
Component  3  is  also  responsible  for  the  crosscutting  functional  area  of  Education, 
Outreach,  and  Training  Coordination  (EOTC). 

Mr.  Merrill  began  his  career  in  1977  as  an  employee  of  the  U.S.  Army  Engineer 
District,  Tulsa.  In  1979  he  transferred  to  a  structural  engineering  position  with  the 
U.S.  Army  Engineer  District,  Kansas,  where  he  worked  in  the  Reservoir  Structures  Section.  In  1984  he 
transferred  to  the  U.S.  Army  Engineer  Waterways  Experiment  Station,  now  the  Engineer  Research  and 
Development  Center,  where  he  worked  on  the  computer-aided  structural  engineering  project,  guidance 
update  programs,  and  performed  complex  analyses  for  various  Corps  offices.  He  became  Chief  of  the 
Computational  Science  and  Engineering  Branch  (CSEB)  of  the  ERDC  InformationTechnology  Laboratory 
in  2002.  The  CSEB  consists  of  20  engineers  and  scientists  with  an  annual  budget  exceeding  $3.5M. 

Mr.  Merrill  earned  his  B.S.  degree  in  civil  engineering  from  Oklahoma  Sate  University  and  received  his 
M.S.  degree  in  civil  engineering  from  Mississippi  State  University.  He  is  a  member  of  the  American  Society 
of  Civil  Engineers. 
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Ready  To  Change  The  World? 

By  Trent  Townsend 


Fresh  out  of  college,  most  new  graduates  are  ready  to 
tackle  the  world  with  breathtaking,  revolutionary  ideas. 
However,  seldom  does  this  phenomenon  actually  take 
place.  More  often  than  not,  new  graduates  take  the 
customary  entry-level  position  in  hopes  of  moving  up 
the  corporate  ladder. . .  at  which  point  they  can  eventu¬ 
ally  reshape  the  world  in  which  we  live.  The  mindset  of 
the  future  working  force  is  not  so  hard  to  understand: 
they  have  just  spent  the  majority  of  their  lives  and 
countless  dollars  preparing  for  this  moment.  Everyone 
has  always  told  them  college  is  a  necessity  to  be 
successful,  so  why  would  this  dream  not  crystallize  into 
such  alleged  success?  The  problem  is  that  college  does 
not  teach  us  all  we  need  to  know  but  rather  prepares  us 
to  learn  while  on  the  job.  We  are  given  the  basics  and 
indeed  the  necessities  to  be  successful,  but  college 
cannot  nor  should  it  claim  to  provide  the  job  experience 
needed  to  make  an  impact  immediately 

Not  being  too  far  removed  from  this  scenario,  I  am  still 
able  to  relate.  I  now  know  that  employers  want  to  see 
a  degree  to  prove  that  college  graduates  have  the 
potential  to  do  the  job,  but  it  was  not  always  that  clear 
However,  I  was  extremely  fortunate  in  that  I  undertook 
two  internships  while  in  college,  one  of  which  turned 
into  a  more  permanent  position  during  my  undergradu¬ 
ate  education.  After  the  first  internship,  I  completely 
recognized  the  value  in  job  experience  in  addition  to  a 
college  education.  Thus,  when  I  was  ready  to  take  the 
second  internship,  I  knew  what  to  expect  and  what  I 
would  hopefully  gain  from  the  experience. 

This  second  opportunity  to  gain  invaluable  experience 
came  when  I  worked  two  summers  as  a  security  intern 
here  at  the  ERDC  MSRC.  Coming  from  a  program¬ 
ming  background  and  stepping  into  a  new  field  in 
information  assurance,  I  was  what  could  be  described 
as  “green.”  I  knew  of  a  few  basic  ideas  and  principles 
that  information  assurance  encompassed  but  was  not 
ready  to  take  full  hold  of  a  permanent  position  in  this 
field.  However,  the  difference  was  that  I  knew  this 
and  prepared  myself  to  learn  rather  than  to  immediately 
make  an  impact  on  programwide  levels. 

Being  fortunate  to  be  placed  under  knowledgeable  and 
experienced  mentors  who  took  time  out  of  their  sched¬ 
ule  to  answer  my  endless  questions,  I  indeed  gained  the 
experience  I  needed.  I  learned  much  about  the  daily 
activities  I  could  expect  once  I  took  a  full-timeposition 
and  how  to  be  of  more  impact  when  that  time  arrived. 
But  more  than  that,  I  was  able  to  learn  about  such 


Trent  Townsend  (left),  ERDC  MSRC,  and  Nathan 
Necaise,  ERDC  MSRC  Summer  Intern 


things  while  doing  work  that  actually  needed  to  be  done 
rather  than  just  doing  “busy  work”  the  entire  summer 
This  is  where  the  real  motivation  for  learning  is  in¬ 
spired.  Knowing  what  you  are  doing  is  going  to  make  a 
difference  in  at  least  a  small  degree  and  will  not  be 
tossed  aside  when  you  are  gone  makes  a  difference  in 
the  attitude  used  while  approaching  a  task. 

After  having  such  good  experiences  as  an  intern,  I  took 
the  new  opportunity  presented  to  me  when  our  recent 
security  intern,  Nathan  Necaise,  arrived  and  tried  to 
provide  the  same  leadership  to  him.  I  tried  to  make  his 
transition  into  our  ranks  easier  his  work  assignments 
clear  and  meaningful,  and  tried  to  provide  the  needed 
guidance  to  accomplish  those  tasks.  These  were  all 
benefits  I  received.  Not  surprisingly,  I  found  Nathan 
was  similar  to  me  when  I  first  came  here.  He  was 
somewhat  inexperienced  in  the  reality  of  what  informa¬ 
tion  assurance  professionals  encounter  on  a  day-to-day 
basis.  To  Nathan’s  credit,  he  is  a  very  quick  learner  and 
works  very  hard  to  provide  genuine  results  that  can  be 
of  immediate  help.  With  such  qualities,  it  was  not  hard 
for  him  to  learn  new  ways  to  accomplish  tasks,  realize 
why  we  do  things  in  certain  ways,  and  manifest  these 
lessons  in  the  form  of  several  useful  projects.  Keeping 
in  mind  that  he  needed  to  be  challenged  in  ways  that  he 
could  learn  while  providing  benefits  to  the  Center  the 
security  team  collectively  came  up  with  a  set  of  items 
to  which  we  wanted  Nathan  to  be  exposed  in  some 
way.  This  took  shape  in  several  exercises  involving  the 
Perl  scripting  language,  practical  research  involving 
Kerberos  and  CAC  applications  on  Mac  OS  X,  and  a 
research  paper  to  name  a  few .  For  every  task  given  to 
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Nathan  Necaise  (third  from  lef)  with  ERDC  MSRC  Security  Ham  members  (ielt  to  right)  George  Moncrief, 

Trent  Townsend,  and  Charles  Ray 


him,  Nathan  quickly  produced  results  and  even  took  the 
initiative  to  go  beyond  what  we  requested  and  do 
research  on  his  own.  As  the  summer  came  to  an  end, 
he  expressed  to  me  that  he  had  enjoyed  his  time  here 
but  more  importantly  felt  his  benefit  from  the  experi¬ 
ence  had  been  great.  Certainly,  the  gain  was  mutual  for 
the  MSRC. 

To  be  stated  clearly  and  reiterated,  I  cannot  emphasize 
the  importance  of  these  internships  enough.  They  are 
beneficial  for  the  MSRC  in  that  we  get  the  fruits  of 
interns’  time  here,  but  we  also  get  the  privilege  of 


providing  the  experience  and  preparation  needed  for 
future  careers  to  each  of  these  interns.  As  a  research 
and  development  center  for  the  Army,  these  achieve¬ 
ments  should  remain  high  on  our  list  of  objectives.  Not 
only  does  it  make  the  ERDC  MSRC  better,  it  makes  a 
difference  to  interns  who  receive  positive  reinforce¬ 
ment  and  much  needed  job  experience.  If  you  have 
such  an  opportunity  to  mentor  an  intern  in  a  similar 
situation,  keep  in  mind  the  privilege  you  have  just  been 
given.  They  still  may  change  the  world. 


SAME  U.S.  Army  Camp 
Students  Visit  ERDC  MSRC 

by  Rose  J.  Dykes 

The  ERDC  MSRC  participated  in  the  annual  Society 
of  American  Military  Engineers  Engineering  and 
Construction  Camp  held  in  Vicksburg  on  June  11-17, 
2006.  This  1-week  camp  provides  high  school  students 
with  hands-on  experience  in  engineering  and  construc¬ 
tion  activities  and  is  supervised  by  professional  engi¬ 
neers  and  volunteers  from  organizations  in  the  lower 
Mississippi  Valley.  Crewmates  (campers)  are  exposed 
to  knowledge  about  various  choices  in  the  fields 
of  engineering  and  construction  primarily  on  water 
resources. 


Dr.  William  Ward  (far  right),  ERDC  MSRC 
Computational  Science  and  Engineering  Lead 
talks  with  a  group  of  campers  in  the  HPC  Center 
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Users  Group  Conference  2006 

By  Rose  J.  Dykes 

The  DoD  High  Performance  Computing  Modernization  Program  Users  Group  Conference  held  its  16th  annual 
meeting  in  Denver,  Colorado,  on  June  26-29,  2006. Approximately  400  attendees  gathered  to  attend  plenary 
sessions,  computational  project  presentations,  a  poster  session,  and  social  events  offering  the  opportunity  for 
networking,  renewing  old  contacts,  and  following  up  on  questions  not  asked  during  individual  computational  project 
talks. 

The  ERDC  MSRC  participated  by  making  seven  oral  presentations  and  seven  poster  presentations. 


Oral  Presentations: 

Randall  Hand-  “Visualization  of  Time-Dependent, 
Quasi-Streamwise  Vortex  Tubes  in  a  Bubble- 
Laden  Turbulent  Boundary  Layer  over  a  Flat 
Plate” 

Keith  Rappold  -  “UIT  Development:  How  to 
Access  HPC  Resour  ces  Using  the  UIT  Web 
Service” 

Paul  Adams  -  “Parallel  Visualization  of  Time- 
Dependent  Data  Sets  on  a  Tiled  Display” 

Tom  Biddlecome  -  “Volume  Rendering  on  Graph  - 
ics  Processors” 

Dr.  Fred  Tracy  -  “Accuracy  and  Performance 
Testing  of  Three-Dimensional  Unsaturated  Flow 
Finite  Element  Groundwater  Programs  on  the 
Cray  XT3  Using  Analytical  Solutions” 

Dr.  Ruth  Cheng  -  “Development  of  a  Parallel 
Coupler  for  a  Watershed  Model” 

Dr.  Jeff  Hensley  -  “An  Overview  of  the  Capabili  - 
ties  and  Performance  of  I/O  on  the  Cray  XT3 
Using  the  Lustre  File  System” 


Poster  Presentations: 

Dr.  Paul  Bennett  -  “Targeting  CCM-,  CEA-,  and 
CSM-Based  Computing  to  Specific  Architectures 
Based  Upon  HPCMP  Systems  Assessment” 

Patti  Duett-  “ezHPC:  Simplifying  Access  to 
ERDC  MSRC  HPC  Resources” 

Randall  Hand  and  Paul  Adams  -  “ezVIZ  -  An  Open- 
Source,  Cross-Platform  Visualization  Tool” 

Amanda  Hines  -  “Parallel  XMDF :  A  Portable 
Parallel  I/O  Format” 

Keith  Rappold  -  “ezHPC  Security  Internals” 

Renee  Mullinax,  Dean  Hampton, Abdul  Mohamed, 

John  Mason,  and  Patti  Duett  -  “OKC:  A  Look  at  the 
New  PET  Online  Knowledge  Cente  r” 

Trent  Townsend,  George  Moncrief,  and  Scotty  Swillie  - 

“ezHPC  Security  Architecture” 


Tuesday  night  Poster  Session — ERDC 
posters  displayed  on  either  side  of  this 
walk-through  area 


32 


ERDC  MSRC  Resource,  Fall  2006 


(Left  to  right)  Barbara  Rooks-Jackson,  Executive 
Assistant,  and  Jo  Ann  Clark,  Staff  Assistant,  both  from 
Office  of  U.S.  Senator  Thad  Cochran  (R-MS);  John  E. 
West,  ERDC  Information  Technology  Laboratory  Acting 
Deputy  Director;  and  Dr.  James  Houston,  ERDC 
Director,  September  12 


(Left  to  right)  Dr.  Houston;  Dr.  Valerie  J.  Gawron, 
Technology  Fellow,  General  Dynamics,  Fairfax,  Vrginia; 
Dr.  Loretta  A.  Moore,  Associate  Professor  and  Chair, 
Department  of  Computer  Science,  Jackson  State 
University;  and  Greg  Rottman,  ERDC  MSRC  Acting 
Director,  September  1 


(Left  to  right)  Dr.  Doug  Post,  Chief  Scientist,  DoD  High 
Performance  Computing  Modernization  Program, 
Arlington,  Virginia;  Dr.  Deborah  Dent,  Acting  Director, 
ERDC  Information  Technology  Laboratory;  and 
Dr.  Robert  Maier,  Assistant  Director,  ERDC  MSRC, 
August  28 
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Paul  Adams  (right),  ERDCMSRC  Scientific 
Visualization  Lead,  demonstrating  a  project  on 
RenderWall  for  Elwyn  Darden,  Special  Assistant  to  the 
Assistant  Secretary  of  the  Army  (Civil  Works), 
Washington,  D.C.,  and  Natalie  Darden,  August  24 


LTG  Carl  A.  Strock  (far  left)  walks  through  Joint 
Computing  Facility  as  Dr  Jeffery  P.  Holland  (second 
from  right),  ERDCActing  Deputy  Director  and  John 
West  (far  right)  discuss  supercomputing  infrastructure, 
July  18 


(Left  to  right)  Paul  Adams;  Dr.  Guillermo  Riveros,  ERDC 
ITL;  Dr.  Domingo  Rodriguez,  Department  of  Civil 
Engineering,  and  Dr.  Ismael  Pagan,  Department  Head, 
University  of  Puerto  Rico,Mayaguez,  August  10 


U.S.  Representative  Bennie  Thompson  (D-MS)  (center), 
discusses  Homeland  Security  with  Dr.  Holland  (left)  and 
John  West  (right),  July  3  (photograph  courtesy  of 
Marsha  Gay) 
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Below  is  a  list  of  acronyms  commonly  used  among  the  DoD  HPC  community.  These  acronyms  are  used  through¬ 
out  the  articles  in  this  newsletter. 


AFIT 

Air  Force  Institute  of  Technology 

AMR 

Adaptive  Mesh  Refinement 

API 

Application  Programming  Interface 

ASC 

Aeronautical  Systems  Center 

CFD 

Computational  Fluid  Dynamics 

CHL 

Coastal  and  Hydraulics  Laboratory 

CPOC 

Component  Point-of  Contact 

CPU 

Central  Processing  Unit 

CSEB 

Computational  Science  and 
Engineering  Branch 

CSM 

Computational  Structural  Mechanics 

CTA 

Computational  Technology  Area 

CWO 

Climate/Weather/Ocean  Modeling 
and  Simulation 

DAC 

Divide  and  Conquer 

DMS 

Data  Management  System 

DoD 

Department  of  Defense 

EOTC 

Education,  Outreach,  andTraining 
Coordination 

EQM 

Environmental  Quality  Modeling  and 
Simulation 

ERDC 

Engineer  Research  and  Development 
Center 

FAPOC 

Functional  AreaPoint-of-Contact 

FAQs 

Frequently  Asked  Questions 

FMS 

Forces  Modeling  and  Simulation 

GBytes 

Gigabytes 

GSL 

Geotechnical  and  Structures 
Laboratory 

GUI 

Graphical  User  Interface 

HPC 

High  Performance  Computing 

HPC2 

HPC  Collaboratory 

HPCMP 

High  Performance  Computing 
Modernization  Program 

HPCMPO 

HPCMP  Office 

ITL 

Information  Technology  Laboratory 

MSRC 

Major  Shared  Resource  Center 

MSU 

Mississippi  State  University 

NAVO 

Naval  Oceanographic  Office 

NDBC 

National  Data  Buoy  Center 

PET 

User  Productivity  Enhancement 
and  Technology  Transfer 

PFLOPS 

One  Thousand  TFLOPS 

R&D 

Research  and  Development 

SAME 

Society  ofAmerican  Military 
Engineers 

SHAMRC 

Second-order  Hydrodynamic 
Automatic  Mesh  Refinement  Code 

SOAP 

Simple  Object  Access  Protocol 
(XML  protocol) 

S&PS 

Survivability  and  Protective  Structures 

TFLOPS 

Trillion  Floating-Point  Operations 
per  Second 

TMax 

Temporal  Maxwell  solver 

UIT 

User  Interface  Toolkit 

USAF 

United  States  Air  Force 

WIS 

Wave  Information  Study 
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For  the  latest  on  training  and  on-line  registration,  one  can 
go  to  the  User  Producti  vity  Enhancement  and  Technology 
Transfer  (PET)  Online  Kno  wledge  Center  Web  site: 

https  ://okc.  erdc.  hpc.mil 

Questions  and  comments  may  be  directed  to  PET 
at  (601)  634-3131,  (601)  634-4024,  or 
PET-Training@erdc.  usace.  army.mil 


ERDC  MSRC  Resource 
Editorial  Staff 


Chief  Editor/Technology  Transfer  Specialist 

Rose  J.  Dykes 

Visual  Information  Specialist 

Betty  Watson 


ERDC  MSRC  HPC  Service  Center 
Web  site:  www.erdc.hpc.mil 
E-mail:  msrchelp@erdc.hpc.mil 
Telephone:  1-800-500-4722 


The  ERDC  MSRC  welcomes  comments  and  suggestions  regarding  the  Resource  and  invites  article  submissions. 

Please  send  submissions  to  the  above  e-mail  address. 

The  contents  of  this  publication  are  not  to  be  used  for  advertising,  publication,  or  promotional  purposes.  Citation  of 
trade  names  does  not  constitute  an  official  endorsement  or  approval  of  the  use  of  such  commercial  products. 

Any  opinions,  findings,  conclusions,  or  recommendations  expressed  in  this  publication  are  those  of  the  author(s) 

and  do  not  necessarily  reflect  the  views  of  the  DoD. 

Design  and  layout  provided  by  theVisual  Production  Center,  Information  Technology  Laboratory,  U.S.  Army 

Engineer  Research  and  Development  Center. 


Approved  for  public  release;  distribution  is  unlimited. 


m 

SSS^SSil 


